Live Web

Live Web Final - Drawing Melody

in

Online app here: www.manxue-wang.com

I like painting and drawing and I'd like to discover more interactive way of drawing. When I was a middle school student, I remember I was always drawing in the class, even exchange each other's notebooks with my friends, and continue their drawing. I think it will be really fun if I can create an online drawing board that everyone can collaborate drawing together.

After decided making a drawing app online, I start to think about fun interact way which can make the drawing experience more interesting. I think about melody. Music is also a beautiful thing in our life, if I can combine this two elements together, It will be really fun. 

I mapping the piano key to the canvas, and mapping the frequency from low to high based on the canvas width.

When user drawing on the canvas, the notes will start to play, users can play the sound in a note or in a scale by changing their drawing. After that, their drawing will affect by the gravity and wind power in this world, every time the drawing collide the canvas edge, sound play. It like echo, after user drawing the sound, it will have some random echo to generate beautiful harmony witch may give the user some surprise.

The app can be played by multiple people online. users can also see what other users drawing and heard the sound others made in real time.

I'm using P5 to make the drawing function , sound generation and physical effect, using socket.io , Node.js and Express.js to made the server side and make the drawing board can be played by multiple user in real time.

 

 

 

Live web midterm proposal

in

For the midterm, I have bunch of ideas. At first I want to do some thing with face tracking and Video chatting. I think it will be interesting if you talk with familiar friends with unfamiliar faces, and it may also affect the way and the things you are talking about.  

Here are some ideas,  for the fifth idea, for example, I'd like to use face tracking library to track people 's face, and detect users' emotion score during the conversation. And also calculate how long he's speaking. If he speak too much or he's getting angry, his head will change to a wolf head or a tiger head. If he has a very happy mood and like listening to others, his face will change to a rabbit or other kind animal's head. I want to use a visual way to translate people's relationship during a conversation. Sometimes we may not have any ideas that we are too ambitious or a little bit impolite in a conversation. This application may help us correct our attitude and change bad emotions in a conversation and help building healthy relationship.

The tricky part for this idea is face tracking won't be reliable all the time, sometime it may have wrong result, and it will affect the experience. And another part is there is noway to calculate people's speaking time. But I like this idea so much..

Another idea I like so much is I want to build a collaborate drawing game, and everything you drew on the canvas can generate notes(music), Other user's drawing elements can have interaction with yours. Every elements in that world have rigid body and have gravity and may also collide with other elements. Every time they collide each other, sound generated.